TokyoTech's TRECVID2007 Notebook

For the high-level feature extraction task, we use visual features (visual words of keyframe images and motion features), and do not use any audio information. Maximum entropy models [1] are employed to model these visual features. Because there was a material mistake in our submission, the inferred Average Precisions of our runs was almost zero. Therefore, in this notebook, we also show the results evaluated by ourselves with the truth judgments file (qrels file). Figure 1 briefly shows our visual feature extraction process. From each keyframe, we first extract Affine regions, then describe each of the regions with a SIFT descriptor. We quantize the SIFT descriptor using a tree-cluster codebook. We call this quantized descriptor “visual word”. In addition to the visual word itself, we combine the visual word and motion information of the corresponding Affine region to get a “motion feature”. Finally, we use the number of occurrences of the visual words and the motion features to construct a feature vector for a maximum entropy model.